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Introduction 


Introduction 

^ .1 The toolkit has been developed by the profession and the two training universities of 

Dundee and Strathclyde to support self-evaluation. Educational psychologists from across 
Scotland, representing all professional levels, have been directly involved in the 
consultation and development of this document. 

The toolkit has been designed to provide a more detailed look at self-evaluation. It 
outlines a systematic approach for educational psychology services to use when 
evaluating the effectiveness with which they deliver their services. 

Evidence produced through the use of this or other evaluation models can contribute to 
overall evaluations. For example, evidence produced through the use of models such as 
Charter Mark can contribute to evaluations made using measures and indicators within 
this framework and vice versa. Many services in the past have used the Psychological 
Service Quality Assurance Performance Indicators. The new framework has built on 
previous models and taken account of recent developments relating to quality assurance. 

The process of self-evaluation 

1 .2 The process of self-evaluation helps services to: 

• recognise the positive impact their work has on stakeholders for example, children and 
young people, parents, families, educational staff and partner agencies; 

• highlight levels of service which need to be maintained or where improvement is 
needed and where they should be working to achieve excellence; 

• identify what they can do to make things better for stakeholders; and 

• inform stakeholders about the quality of services in the area. 

The process of using self-evaluation for improvement is based around three key 
questions; 

How are we doing? 

How do we know? 

What are we going to do now? 

The following provides more detailed information about how a service might go about 
answering these three questions. 

How are we doing? 

1 .3 Practitioners have always reflected on the quality of the services they provide. By working 
together to gather information to evaluate the impact of services, educational 
psychologists can come to a shared view of how well they are doing and how they can 
make things better. To ensure the quality of provision is maintained and improved, 
service staff also need to evaluate how effectively services are delivered and managed, 
and how well they plan for improvement. 
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Making such evaluations is dependent on a shared understanding between service 
providers and stakeholders of what constitutes high-quality outcomes and processes. The 
quality and performance indicators in this document set out to support the development 
of such an understanding by all those concerned with delivering or evaluating 
educational psychology services. Answering the question How are we doing? requires 
services to summarise their impact on external and internal stakeholders, and indeed the 
community as a whole. 

How do we know? 

1 .4 Self-evaluation involves: 

• a broad view of performance across the six high-ievel questions 

• a cioser iook at particular aspects of work. 

Forming a broad view 

1 .5 Practitioners within a particular service can use the quality indicators to form a broad 
view of quality across their service. Using evidence that has been gathered in the normal 
course of their work, they can identify broad strengths and weaknesses. This will enable 
them to make an immediate evaluation of areas of major strength, or areas where more 
attention is required. 

Taking a closer look 

1 .6 Often it may not be manageable or even helpful to try to evaluate every aspect of the 
service in an in-depth way at the same time. In order to have a greater understanding of 
the effectiveness of particular aspects of practice, managers and practitioners can take a 
closer look at them. The stimulus to take a closer look could derive from a range of issues 
that includes: 

• the decision of a small group of practitioners to follow up a particular issue or area on 
which they have been working in order to find ways to evaluate and improve what 
they are doing; 

• an area of priority identified during the broad view of self-evaluation; 

• a national priority or a local improvement objective; 

• a survey of the views of stakeholders in an area; 

• the outcomes of an inspection or inquiry; and 

• research findings which have implications for the quality of the service. 

This means that specific groups of practitioners can also carry out self-evaluation by 
asking themselves focused questions such as: 

• How are educational psychologists contributing to improving outcomes for children 
and young people looked after and accommodated by the local authority? 

• How well is the service complying with legislation and responsiveness to guidance and 
codes of practice? 


It could also focus on specific themes by asking such questions as: 

• How well does the service consult with children and young people, and take account 
of their views in the delivery of the service? 

• How effectively do educational psychologists work in teams? 

By encouraging practitioners to structure their discussion of, and reflection on, their 
work, the quality and performance indicator framework can support improvement by 
individuals and teams as well as at operational and managerial levels. 


How do we gather evidence? 

1 .7 There are a number of sources of evidence which can inform a service about how well 
the needs of stakeholders are being met and what differences are being made. By using 
range of indicators and sources of evidence, an holistic view of quality can be 
determined. There are four key sources of evidence from which evaluations can 
ultimately be made. These are: 

• performance data 

• relevant documentation 

• stakeholders views and feedback 

• direct observations of practice. 



1 .8 These sources of evidence are complementary. A single source may not provide enough 
evidence to enable a reliable or robust evaluation to be made. The principle has been 
tried and tested over many years by independent external evaluators and entails the 
scrutiny of one source of evidence, backed up by another and corroborated by other 
lines of enquiry. An example would be an evaluation based on an analysis of the 
educational psychology service development plan, the implementation of which was 
then fleshed out in discussion with senior educational psychology staff and further 
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corroborated in discussion with stakeholders such as parents, carers and families. 

Through such a process of robust self-evaluation it would be possible to monitor the 
outcomes of putting policy into practice, and, by doing so, evaluate the benefits of 
policy in meeting the needs of stakeholders. 

Performance data 

1 .9 Examples of performance data would be statistical information relating to local and 
national objectives. 

Relevant documentation 

1 .1 0 Examples of relevant documentation could be a statement of the service's vision, values 
and aims, service development plans, standards and quality reports. 

Stakeholders' views 

1 .1 1 Information can be collected systematically when stakeholders are accessing and using a 
service, or at the end their involvement with a service. Services should also have 
procedures for surveying stakeholders' views using questionnaires or focus groups. 

1 .12 Whatever approach is used, gathering information from stakeholders is an essential part 
of the self-evaluation process. Without it, services will find it very difficult to understand 
the impact of their work on stakeholders. It is almost impossible to answer the 
high-level questions with any degree of confidence without including the views of 
stakeholders. 

Direct observation 

1 .1 3 The educational psychology service is embedded and managed within local authority 
systems and delivers its statutory functions within that local context. The service works 
collaboratively to support and empower others to improve outcomes for stakeholders. 
This approach is supported by evidence-based practice that recognises the effectiveness 
of a process of collaborative, contextual assessment and intervention. 

1 .14 To measure the impacts and outcomes of the involvement of the service, it is necessary 
to consider the relationships the service has and the context in which its services are 
delivered. The impact of the service is concerned with the immediate and current 
experiences of stakeholders: it has a focus on customer service. Outcomes are 
longer-term and largely statistical measures relating to improvement programmes and 
achievement data. Outcomes are concerned with the service's success in achieving its 
objectives and also its contributions to the education authority's objectives. 

1 .1 5 The service operates at the different levels of delivery - on behalf of the individual, 
school/provision, local authority or within a national context - at different times. For 
example, with regards to individual case work, sometimes the service will be in direct 
contact with a child or young person and, at other times, the service will work through a 
parent or teacher to improve outcomes for children and young people. The impact of 
the psychological service can be direct or indirect. The work of the service can be directly 
observed in a range of contexts as illustrated opposite: 



Individual 

Family 

Educational 

Provision 

Authority 

National 

Consultation 
and Advice 

Individual 
discussions, 
contributions to 
individualised 
educational 
plans, or 
co-ordinated 
support plans. 

Home visits, 
parent and 
review 
meetings. 

joint working 
with staff, 
provision of 
advice on 
programmes, 
contributions to 
strategic 
planning and 
policy advice. 

Contributions to 
strategic 
planning on 
behalf of senior 
managers. 

Contributions to 
strategic 
planning at a 
national level. 

Assessment 

Observation of 
a range of 
assessments in 
different 
contexts. 

Observation of 

parent/child 

interaction. 

Contributions to 
school's 
assessment 
policy and 
procedures. 

Contributions to 
the authority's 
assessment 
policy and 
procedures. 

Contributions to 
national 
assessment 
policy and 
procedures. 

Intervention 

Observation of 

implementation 

of behaviour 

management 

programmes 

and therapy 

sessions. 

Observation of, 
and assistance 
with, the 
implementation 
of a joint action 
plan developed 
for use at home 
and school. 

Observation of, 
and 

contributions 
to, whole- 
establishment 
interventions 
(e.g. anti- 
bullying 
strategies, 
playground 
behaviour, 
raising 
attainment), 
assessment 
arrangements 
for candidates 
with disabilities 
and/or 
additional 
support needs, 
curricular 
innovation/ 
initiatives, 
supporting 
specialised 
college 
placements. 

Observation of, 
and 

contributions 
to, authority- 
wide 

interventions 

(e.g. raising 

attainment, 

alternatives to 

exclusion, 

promoting 

social inclusion, 

promoting 

resilience, 

resource 

allocation). 

Observation of, 
and 

contributions 
to, national 
interventions 
(e.g. anti- 
bullying 
strategies 
enhancing 
achievement, 
citizenship 
programmes, 
promoting 
social inclusion). 

Professional 
Development 
and Training 

Observation of 
training for a 
young person 
with additional 
support needs. 

Observation of 
participation in 
design, 

implementation 
and evaluation 
of staff training, 
dissemination of 
evidence-based 
practice to 
groups of 
parents or 
carers. 

Observation of 
participation in 
design, 

implementation 
and evaluation 
of staff training, 
dissemination of 
evidence-based 
practice in 
single or groups 
of educational 
provision. 

Observation of 
participation in 
design, 

implementation 
and evaluation 
of authority 
wide staff 
training 
initiatives, and 
dissemination of 
evidence-based 
practice. 

Observation of 
participation in 
design, 

implementation 
and evaluation 
of national staff 
training 
initiatives and 
dissemination of 
evidence-based 
practice. 
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Individual 

Family 

Educational 

Provision 

Authority 

National 

Research and 

Evidence of. 

Evidence of. 

Evidence of. 

Evidence of. 

Evidence of. 

Strategic 

and 

and 

and 

and 

and 

Development 

participation in. 

participation in. 

participation in. 

participation in. 

participation in. 


design. 

design. 

design. 

design. 

design. 


impiementation 

implementation 

impiementation 

implementation 

impiementation 


and evaiuation 

and evaiuation 

and evaiuation 

and evaiuation 

and evaiuation 


of action 

of action 

of action 

of authority 

of nationai 


research 

research 

research 

wide action 

action research 


projects 

projects 

projects in 

research 

projects (e.g. 


involving 

invoiving 

singie or groups 

projects (e.g. 

restorative 


individuai 

parents, carers 

of educational 

eariy 

practices, video 


chiidren. 

and famiiies 

provision (e.g. 

intervention. 

interactive 



(e.g. person- 

promoting 

strategies 

guidance). 



centred 

positive peer 

raising 




planning 

approaches). 

reiationships). 

attainment). 



What are we going to do now? 

1 .16 It is important, when undertaking any form of self-evaluation, to keep focused on the 

end purpose, which is improving outcomes for stakeholders. The answer to. What are we 
going to do now? must therefore always be a plan for action, which will make a positive 
difference to the stakeholders who use the service. 


Pianning for improvement 

1 .1 7 Effective and robust self-evaluation provides a strong basis for good planning. Such 

planning for the service takes place within the framework of the authority's strategic and 
operational plans. The service will have its own improvement plan or equivalent. 
Self-evaluation can support these planning processes and ensure that plans are built on 
robust, evidence-based knowledge of the quality of impact and outcomes. The results of 
the planning will then be made publicly available in the standards and quality report. 

Action for improvement 

1.18 Whether, and how, the outcomes of self-evaluation are reported depends on the purpose 
and the level of the work done. The purpose of reporting should help decision-making 
about how to make future improvements, and also inform stakeholders. 

1.19 The following examples outline a range of reports which result from self-evaluation 
activity. 

• A service decides to find out the views of young people who have been directly 

involved with its work. A group of young people are selected to be part of this project. 
Parental permission is sought before the young people are approached by the service. 
Permission and agreement are also sought directly from the young people. A series of 
focus groups with young people in various establishments takes place. A list of ten key 
questions are used in the focus groups. The results of the questions are analysed. The 
findings are used to improve the guidelines for direct involvement with young people. 
The service also feeds back the results of the evaluation to the stakeholders involved 
and other interested parties. 


• A service decides to evaluate the contribution it makes to multi-agency meetings and 
to explore the impact a pilot collaborative approach has on children and their families. 
This is done using focus groups including education staff, parents and children. 
Questionnaires are also sent out to all schools. As a result of a positive self-evaluation, 
the service uses the approach across the whole service. The service also feeds back the 
results of the evaluation to the stakeholders involved and other interested parties. 

• A service evaluates its contribution to national and local authority priorities by building 
these priorities into its development plan. It takes feedback from all stakeholders, 
including education service management, on the tasks achieved and quality of the 
input provided by the service to a range of local authority working groups and 
initiatives. The service publishes impacts and outcomes in its yearly standards and 
quality report. 

1 .20 As a service works across a wide range of stakeholders, meeting the needs of stakeholders 
can involve a number of professional and user groups. It is important, therefore, that the 
outcomes of self-evaluation and plans for improvement feed into the appropriate 
planning structures. Depending on the work undertaken, the outcomes may result in 
priorities being included in future plans. The following list gives some examples: 

• individualised educational programme for a child or young person; 

• support service or school development plan; 

• plan at cluster or neighbourhood level; 

• local health improvement plan; 

• education authority improvement plan; and 

• children's services plan. 

1 .21 Whatever the planning structure(s) for taking forward improvement, it will help if a 
manageable number of priorities is selected for which a service can identify specific, 
achievable, measurable and time-bound targets. Even where a service can see how 
improvements can be made across a number of aspects, the service may wish to focus 
on those of greatest concern. It will make more impact on stakeholders if a manageable 
number of priorities are taken forward effectively. In some cases, small changes in 
practice identified by practitioners can have a significant positive impact for many 
different stakeholders. It is not always necessary for self-evaluation to result in major 
changes or reviews of practice. Often these changes are part of day-to-day professional 
working which focuses on continuous improvement. 
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1 .22 The process of self-evaluation as described above is summarised by the following 
diagram. 



The toolkit has been designed to support educational psychology services and practioners in 
addressing these questions. In the first column headed How are we doing? questions have 
been developed by the profession to explore this theme. In the second column headed How 
do we know? examples of four main sources of evidence, have been outlined: 

• performance data; 

• relevant documentation; 

• stakeholders' views and feedback; and 

• direct observation. 


In Appendix II, examples of relevant documentation for quality indicators have been listed for 
consideration. 

In the third column headed What are we going to do now? a series of statements are provided 
for a service to review. By examing these statements, the service should be able to assess how 
it can improve the overall quality of the educational psychology service. 

It is essential that self-evaluation should not be viewed as a discrete summative task but one 
that is formative and will allow the service to improve outcomes for stakeholders. To achieve 
this end, there is a final sub-section in the third column headed We will? By completing this 
sub-section in an open and robust fashion, the service should be assisted in formulating an 
action plan for improvement (see figure 1). 


HOW ARE WE DOING? 

HOW DO WE KNOW? 

WHAT ARE WE GOING TO DO NOW? 



To ensure that: 
We will: 




Part A — Support for self-evaluation 


A 


io_n 


what key outcomes have we achieved? 

(KEY AREA 1: KEY PERFORMANCE OUTCOMES) 

PI 1.1 Improvement in performance 

Themes: 

• Performance data and measures showing trends over time 

• Performance against national, local authority and psychological service aims, objectives and 
targets 

Key Features 

This indicator evaluates continuous and sustainable improvement against national and local 
objectives such as those contained in the authority's Children's Services Plan. Examples of 
performance data and measures might include measurable outcomes from the authority's 
strategic and operational plans. 

Performance will also be measured against objectives within the services improvement plan. It 
will include the contribution of the educational psychology service in meeting local targets for 
education and care in the Children's Services Plan and other plans. Examples of this could 
include: 

• achievement of targets for children and young people; 

• improvements following service reviews relating to Best Value; and 

• outcomes of research and development initiatives which have had an impact on the 
learning and wellbeing of children and young people. 
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The term 'looked after' in this report includes all children looked after or looked after and accommodated by the council. 
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what key outcomes have we achieved? 

(KEY AREA 1: KEY PERFORMANCE OUTCOMES) 

PI 1 .2 Fulfilment of general statutory duties 

Themes: 

• Financial performance 

• Compliance with legislation, and responsiveness to guidance and codes of practice 

Key Features 

Evaluation of financial performance will be based on financial data and measures derived from 
local Best Value reviews, and from the authority budget construction and management 
systems. 

Compliance with statutory requirements relates to legislation and codes of practice such as the 
Education (Additional Support for Learning) (Scotland) Act 2004 and the Standards In 
Scotland's Schools etc. Act 2000. 
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A 



How well do we meet the needs of our stakeholders? 


(KEY AREA 2: IMPACT ON STAKEHOLDERS) 
Ql 2.1 Impact on children and young people 


Themes: 


• Quantitative and qualitative data that demonstrate the extent to which children and young 
people are: 

- included and participating 

- achieving and attaining 

- progressing. 

• The extent to which children and young people report that the support they had received 
from the service contributed positively to their educational experiences and enabled them 
to become: 

- successful learners, confident individuals, responsible citizens and effective contributors 

- safe, nurtured, healthy, achieving, active, respected and responsible and included. 

Key Features 

This indicator relates to the impact of the service on children and young people, focusing in 
particular on their current and recent experiences. Some examples of appropriate sources of 
evidence are given below, however they are not considered to be comprehensive or 
prescriptive. 

The first theme draws on quantitative and qualitative data and evidence gathered from direct 
observation, documentation and discussions with stakeholders. Examples may include: 

• levels of exclusion from school; 

• the extent to which the needs of children and young people are met. These children and 
young people include those from the lowest-performing 20%, those who are looked after 
by the authority and those children and young people from minority ethnic and 
gypsy/traveller families. Performance will be indicated by the achievement of individual 
targets for learning and the acquisition of social skills particularly at the time of transition; 

• achievement of customer service awards such as Charter Mark; 

• correspondence and contact with the service, including complaints and compliments and 
the way these have been managed; 

• access to the service, including innovative approaches to encouraging involvement; and 

• evidence from HMIE inspection reports relating to specific aspects such as the quality of 
support for learning and achievement. 
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The second theme deals with the views as reported in responses to questionnaires, surveys, 
focus groups and in unsolicited comments. These responses provide evidence of the extent of 
stakeholder satisfaction with the service and may cover aspects such as: 

• attitudes to involvement with the service; 

• engagement in informal and formal contact with the service which enables children and 
young people to achieve their full potential, keep themselves healthy and themselves and 
others safe; 

• the extent to which they feel valued and supported by service staff; and 

• opportunities to express their views in terms of shaping services, setting personal objectives 
and influencing planning for their future. 

Evaluation should take both themes into account to produce a considered view of the overall 
impact on children and young people. 


A 
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How well do we meet the needs of our stakeholders? 

(KEY AREA 2: IMPACT ON STAKEHOLDERS) 

Ql 2.2 Impact on parents/carers and families 

Themes 

• Quantitative and qualitative data that demonstrate the extent to which parents/carers and 
families are: 

- treated equally and fairly 

- satisfied with the quality of service 

- included and engaged in planning and decision-making. 

• Extent to which parents/carers and families report that they are: 

- treated equally and fairly 

- satisfied with the quality of the service 

- included and engaged in planning and decision-making. 

Key Features 

This indicator relates to the impact of the service on parents and carers and seeks to obtain 
their views of the quality of service received by their children, with a particular focus on their 
current and recent experiences. Some examples of appropriate sources of evidence are given 
below, however, they are not considered to be comprehensive or prescriptive. 

The first theme draws on quantitative and qualitative data and evidence from direct 
observation, documentation and discussions with other stakeholders. Examples may include: 

• evidence of involvement in planning in relation to their children; 

• evidence of involvement in decision-making in relation to their children; 

• attendance at family-group conferences and joint assessment team meetings; 

• the extent to which the needs of parents, carers and families are met; 

• correspondence and contact with the service, including enquiries, complaints and 
compliments and the way these are managed; 

• access to services, including innovative approaches to encouraging involvement; and 

• evidence from HMIE inspection reports which relate to parents and carers. 

The second theme deals with the views of parents and carers as reported in responses to 
questionnaires, surveys, focus groups and in unsolicited comments. These responses provide 
evidence of the extent of their satisfaction with the service and may cover aspects such as: 




• access to, and contact with, service staff; 

• involvement in planning and decision making on behalf of their children in both informal 
and formal contexts; 

• the quality and range of services provided; 

• the extent to which they feel valued and supported, and are treated fairly, equally and with 
respect; and 

• opportunities to express their views, shape services, and influence outcomes for their 
children. 

Evaluation should take into account both themes and result in a considered view of the overall 

impact on children and young people. 
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A 


26_27 


How well do we meet the needs of our stakeholders? 

(KEY AREA 3: IMPACT ON STAFF) 

Ql 3.1 Impact on staff 

Themes: 

• Quantitative and qualitative data that demonstrate the extent to which staff: 

- are motivated, confident and valued 

- improve their practice through training, development activities 

- have positive experiences of the quality of support from central services and partner 
agencies 

- work effectively in teams 

• Extent to which staff report that they: 

- are motivated, confident and valued 

- improve their practice through training, development activities 

- have positive experiences of the quality of support from central services and partner 
agencies 

- work effectively in teams 

Key Features 

This indicator relates to the impact of the service on those who are managed within the 
service, as well as their views of the quality of service within the authority. It considers the level 
of ownership by staff in the targets set by the service and their active involvement in service 
development. It includes the extent to which the service supports staff to continue their 
professional development which reflects both their personal and professional as well as service 
goals. It looks for any opportunities that exist within the service for leadership and career 
development for individual staff members. It also looks at team working and at any 
opportunities that are available to develop as a service team. Finally, it examines the quality of 
support which the service receives from central services. 

Examples of appropriate sources of evidence are given below. They are considered to be 
neither comprehensive nor prescriptive. 

The first theme draws on quantitative and qualitative data and evidence from direct 
observation, documentation and discussions with stakeholders. Examples may include: 

• rates of attendance at meetings and forums; 

• rates of participation in career review; 

• rates of participation in training and development and qualifications gained; 


• rates of response to consultation; 
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• evidence of involvement and empowerment; 

• the extent to which the individual support needs of staff are met; 

• achievement of awards such as Scotland's Health at Work and Investors in People; 

• access to services, including innovative approaches to encouraging involvement; 

• levels of absence, turnover and recruitment; 

• recognition schemes; and 

• benchmarking data. 

The second theme deals with the views of staff as reported in response to questionnaires, 
surveys, focus groups and in any unsolicited comments received. These responses may provide 
evidence of the level of staff satisfaction and may cover aspects such as: 

• the quality and range of services provided by the authority; 

• career review; 

• career development; 

• training and development, including leadership training, work shadowing and 
secondments; 

• peer and line management relationships, including team working; 

• the extent to which staff feel valued and supported and are treated fairly, equally and with 
respect; 

• conditions of work, facilities and services; and 

• communication and opportunities to express their views and to shape and improve 
education services. 

Evaluation should take into account both themes to produce a considered view of the overall 
impact on staff. 
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How well do we meet the needs of our stakeholders? 

(KEY AREA 4: IMPACT ON THE COMMUNITY) 

Ql 4.1 Impact on the local community 

Themes: 

• Quantitative and qualitative data that demonstrate the extent to which support services, 
educational provisions and the local community are: 

- involved in joint planning regarding service priorities 

- satisfied with the quality of the service provided 

- included and engaged with the service in wider developments. 

• Extent to which support services, educational provisions and the local community report 
that they are: 

- supported by the service 

- satisfied with the quality of service provided 

- included and engaged with the service in wider developments. 

Key Features 

This indicator relates to the impact of the service on support services, educational provisions 
and the local community. It also relates to the views of staff regarding the quality of service 
received by children and young people, focusing in particular on their recent and current 
experiences. Some examples of appropriate sources of evidence are given below. However, 
they are not considered to be comprehensive or prescriptive. 

The first theme draws on quantitative and qualitative data, and evidence from direct 
observation, documentation and discussions with other stakeholders. Examples may include: 

• involvement in joint planning and quality assurance activities in relation to the priorities of 
the service; 

• involvement in joint decision-making in relation to children and young people; 

• attendance at conferences and joint assessment team meetings; 

• correspondence and contact with the service, including enquiries, complaints and 
compliments and the way these are managed; 

• access to a wide and comprehensive range of services, which is regularly monitored and 
reviewed; and 

• evidence from HMIE inspection reports which relate to educational provision, child 
protection, and community learning and development. 




The second theme deals with the views of representatives from the local community, heads of 
services and educational provision and their staff as reported in responses to questionnaires, 
surveys, focus groups and in unsolicited comments. These responses provide evidence of the 
extent of their satisfaction with the service and may cover aspects such as: 

• the quality and range of services provided; 

• examples of partnership working, planning and joint decision making on behalf of children 
and young people in both informal and formal contexts; 

• the extent to which service and provision staff feel valued and supported, and are treated 
fairly, equally and with respect; and 

• formal opportunities to express their views, shape services, and influence outcomes for 
children and young people. 

Evaluation should take into account both themes to produce a considered view of the overall 
impact on children and young people. 
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How well do we meet the needs of our stakeholders? 


(KEY AREA 4: IMPACT ON THE COMMUNITY) 
Ql 4.2 Impact on the wider community 


Themes: 


• Evaluations of quantitative and qualitative data that demonstrate the extent to which the 
service: 

- encourages and supports creativity and innovation 

- learns from and adopts leading-edge practice 

- influences wider policy or practice 

- anticipates and responds rapidly and flexibly to change. 

Key Features 

This indicator focuses on the impact the service has on the wider community. It deals with the 
culture of the service in encouraging and supporting creativity and innovation, being proactive 
and open to new ideas. It is about being a service that deals positively with change. It also 
relates to the influence and impact of the service on wider developments. 

The theme draws on quantitative and qualitative data from direct observation, documentation 
and discussions with stakeholders. Some examples of appropriate sources of evidence are given 
below. They are, however, not considered to be comprehensive or prescriptive. Examples may 
include: 

• a range of innovative strategies and programmes that impact beyond the services delivered 
by the service; 

• programmes that have their origins in national or international best practice; 

• staff who have made major contributions to national developments; and 

• anticipation of demographic or social changes with re-allocation of resources and services. 




QUALITY MANAGEMENT IN LOCAL AUTHORITY EDUCATIONAL PSYCHOLOGY SERVICES - SELF-EVALUATION FOR QUALITY IMPROVEMENT 






QUALITY MANAGEMENT IN LOCAL AUTHORITY EDUCATIONAL PSYCHOLOGY SERVICES - SELF-EVALUATION FOR QUALITY IMPROVEMENT 


How good is our delivery of key processes? 

(KEY AREA 5: DELIVERY OF KEY PROCESSES) 

Ql 5.1 Consultation and advice 

Themes: 

• Range and appropriateness of consultation and advisory services meet the needs of all 
stakeholders 

• Staff skill, knowledge and expertise in consulting with and providing advice to stakeholders 

Key Features 

This indicator draws on quantitative and qualitative data from direct observations, discussions 
and documentation including individual case files. Some examples of appropriate sources of 
evidence are given below. However, they are not considered to be comprehensive or 
prescriptive. Examples may include: 

• observing practice directly in the field; 

• attendance at case reviews, and working groups; 

• discussions with service staff and stakeholders; 

• review of individual case files and other service documentation; and 

• the use of case studies to determine service impact. 

The second theme looks at the skills, knowledge base, and expertise of service staff in 
consulting with and providing advice to stakeholders across the authority. It will consider the 
range and frequency of guidance and training for staff in delivering highly effective 
consultation and advisory services. 
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How good is our delivery of key processes? 

(KEY AREA 5: DELIVERY OF KEY PROCESSES) 

Ql 5.2 Assessment 

Themes: 

• Arrangements and range of assessment services to meet the needs of all stakeholders 

• Staff skill, knowledge and expertise in delivering appropriate psychological assessment 

Key Features 

This indicator draws on quantitative and qualitative data from direct observations, discussions 
and documentation including individual case files. Some examples of appropriate sources of 
evidence are given below. However, they are not considered to be comprehensive or 
prescriptive. Examples may include: 

• observing practice directly in the field; 

• discussions with service staff and stakeholders including partner agencies; 

• attendance at case reviews, multidisciplinary meetings, working groups and training events; 

• review of individual case files and other service documentation; and 

• the use of case studies to determine service impact. 

The second theme looks at the skills, knowledge base and expertise of staff undertaking 
assessment across the authority. It will consider the range and frequency of guidance and 
training for staff in delivering a highly effective psychological assessment. 
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How good is our delivery of key processes? 

(KEY AREA 5: DELIVERY OF KEY PROCESSES) 

Ql 5.3 Intervention 

Themes: 

• Arrangements for advising, planning, delivering and evaluating intervention strategies to 
meet the needs of all stakeholders 

• Staff skill, knowledge and expertise in planning and delivering effective psychological 
interventions 

Key Features 

This indicator draws on quantitative and qualitative data from direct observations, discussions 
and documentation including individual case files. Some examples of appropriate sources of 
evidence are given below. However, they are not considered to be comprehensive or 
prescriptive. Examples may include: 

• observing practice directly in the field where appropriate; 

• discussions with service staff and stakeholders including partner agencies; 

• attendance at case reviews, multidisciplinary meetings, working groups and training events; 

• review of individual case files and service documentation; and 

• the use of case studies to determine service impact. 

The second theme looks at the skills, knowledge base and expertise of staff advising, planning 
and delivering interventions across the authority. It will consider the range and frequency of 
guidance and training for staff in delivering and supporting the delivery of highly effective 
psychological interventions. 
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How good is our delivery of key processes? 

(KEY AREA 5: DELIVERY OF KEY PROCESSES) 

Ql 5.4 Professional development and training 

Themes: 

• Range and appropriateness of professional development and training to meet the needs of 
all stakeholders 

• Staff skill, knowledge and expertise in planning and delivering effective professional 
development and training to stakeholders 

Key Features 

This indicator draws on quantitative and qualitative data from direct observations, discussions 
and documentation including individual case files. Some examples of appropriate sources of 
evidence are given below. However, they are not considered to be comprehensive or 
prescriptive. Examples may include: 

• observing, where appropriate, practice directly in the field; 

• discussions with service staff and stakeholders including partner agencies; 

• attendance at case reviews, multidisciplinary meetings, working groups and training events; 

• review of individual case files and other service documentation; and 

• the use of case studies to determine service impact. 

The second theme looks at the skills, knowledge base and expertise of staff advising, planning 
and delivering professional development and training across the service and the authority. It 
will consider the range and frequency of guidance and training for staff in delivering and 
supporting the delivery of highly effective professional development and training. 
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How good is our delivery of key processes? 

(KEY AREA 5: DELIVERY OF KEY PROCESSES) 

Ql 5.5 Research and strategic development 

Themes: 

• Range and appropriateness of the research and development programme to meet service 
and authority priorities 

• Skill, knowledge and expertise of staff in the participation in and undertaking of research 
and development activity 

Key Features 

This indicator draws on quantitative and qualitative data from direct observations, discussions 
and documentation including individual case files. Some examples of appropriate sources of 
evidence are given below. However, they are not considered to be comprehensive or 
prescriptive. Examples may include: 

• observing, where appropriate, practice directly in the field; 

• discussions with service staff and stakeholders including partner agencies; 

• attendance at case reviews, multidisciplinary meetings, working groups and training events; 

• review of individual case files and service documentation; and 

• the use of case studies to determine service impact. 

The second theme looks at the skills, knowledge base and expertise of staff delivering the 
research and development programme across the service and the authority. It will consider the 
range and frequency of support, guidance and training for staff in delivering a high quality 
research and development programme. 
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How good is our delivery of key processes? 

(KEY AREA 5: DELIVERY OF KEY PROCESSES) 

Ql 5.6 Inclusion, equality and fairness 

Themes: 

• Promotion of inclusive practices 

• Evidence of equal opportunities and fairness embedded in all aspects of practice 
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How good is our delivery of key processes? 

(KEY AREA 5: DELIVERY OF EDUCATION PROCESSES) 

Ql 5.7 Improving the quality of services 

Themes: 

• Arrangements for quality assurance and improvement 

• Support and challenge 

• Evaluating outcomes and feedback from stakeholders 

• Planning for improvement and monitoring progress 

• Reporting progress to stakeholders 
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How good is our management? 

(KEY AREA 6: POLICY DEVELOPMENT AND PLANNING) 

Ql 6.1 Policy review and development 

Themes: 

• Range and appropriateness of policies 

• Coherence with council-wide policy 

• Links to vision, values and aims 

• Managing, evaluating and updating policies 

Key Features 

A successful, modern organisation will have a systematic and well-documented approach to 
management. This will be supported by a range of effective policies and advice that inform 
and impact on practice throughout the service, which in turn are linked to the wider policies of 
the authority. These policies provide clear strategic direction and help to ensure consistency in 
practice across the service and improved outcomes for children, young people and families. 

Useful definition 

What is the difference between policies, procedures, and practice guidelines? 

A policy describes what you do and why you do it in that way. A procedure is an articulation of 
the policy at an operational level, i.e. how we deliver the policy in practice. A practice 
guideline can be one type of procedure, e.g. a practice guideline on assessment outlines how 
to apply the service assessment policy in context. 
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How good is our management? 

(KEY AREA 6: POLICY DEVELOPMENT AND PLANNING) 

Ql 6.2 Participation of stakehoiders 

Themes: 

• Involvement in policy development 

• Communication and consultation 

• Active participation in the work of the service 

Key Features 

To fulfil the requirements of Best Value, and to conform to accepted good practice in local 
government, mechanisms need to be in place to link service management decisions to the 
needs of the community. There is an expectation that key stakeholders are actively involved in 
the development of the service and this will require a range of approaches and mechanisms to 
be applied across the service's areas of activity. To complement consultative mechanisms there 
needs to be an effective approach to communication. This will recognise that there are a 
number of different audiences which need to be kept informed about service activities. There 
should be a framework which facilitates effective communication with stakeholders. This will 
require a variety of approaches suited to the audiences concerned and will therefore involve a 
range of communication media. 
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How good is our management? 

(KEY AREA 6: POLICY DEVELOPMENT AND PLANNING) 

Ql 6.3 Operational planning 

Themes: 

• Developing, implementing and evaluating plans 

• Structure and content of plans 

• Use of management information 

• Joint planning with partner organisations and services 

• Planning for sustainability 

Key Features 

The service plan, or its equivalent, will be central to performance management and planning. 
Documents in themselves can only be a partial reflection of effective management and 
planning, since the processes that surround both the development of the plan and its 
implementation are of critical importance. Operational planning should lead to service 
improvement and guide its focus, methodologies and mechanisms for reporting and future 
action. Service planning will be embedded within the service's approach to planning and clear 
links should exist between the Children's Services and other authority plans. 
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How good is our management? 

(KEY AREA 7: MANAGEMENT AND SUPPORT OF STAFF) 

Ql 7.1 Sufficiency, recruitment and retention 

Themes: 

• Identifying and meeting human resource needs 

• Recruitment, appointment and induction procedures 

• Care and welfare 

• Equality and fairness in recruitment and promotion 

• Recognition 

Key Features 

This indicator relates to the service's effectiveness in managing the recruitment, appointment, 
induction and care and welfare of its staff. To provide a high quality service to its stakeholders, 
the service needs a clear understanding of its human resource requirements and effective 
procedures in place to recruit, retain, support and develop its staff to a high level. This will 
require proactive and imaginative recruitment procedures. The principles of equality and 
fairness, together with a commitment to recognise and celebrate achievement, will underpin 
the service's philosophy and practice in its management and support of staff. 
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How good is our management? 

(KEY AREA 7: MANAGEMENT AND SUPPORT OF STAFF) 

Ql 7.2 Deployment and teamwork 

Themes: 

• Appropriateness and clarity of remits 

• Deployment to achieve planned priorities 

• Teamworking 

• Communication and involvement in decision-making 

Key Features 

This indicator is concerned with the effectiveness of individual and team contributions. The 
main asset of any service is its people. In order to achieve its objectives and to implement the 
service plan, staff need to be deployed effectively to respond to the needs of stakeholders and 
to develop effective teamwork and communication. Their work should focus on the 
achievement of the service's planned priorities and improvement objectives whilst also 
responding to key stakeholders' needs. 
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How good is our management? 

(KEY AREA 7: MANAGEMENT AND SUPPORT OF STAFF) 

Ql 7.3 Development and support 

Themes: 

• Processes for staff review and support 

• Training and development 

• Joint training with staff from partner agencies 

Key Features 

This indicator relates to the service's management of its review processes and to the provision 
of training and development opportunities for its staff. It evaluates the service's effectiveness in 
monitoring and reviewing the performance of its staff against agreed criteria, and facilitating 
their personal and professional development. All staff must be able to develop their skills to 
maximum effect in order to achieve continuous improvement in their professional skills. Staff 
have an entitlement to receive a formal programme and record of professional review and 
development (PRD) or CPD. 
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How good is our management? 

(KEY AREA 8: PARTNERSHIPS AND RESOURCES) 

Ql 8.1 Partnership working 

Themes: 

• Evaluations of quantative and qualitative data that demonstrate the extent to which the 
service: 

- engages with, and supports, partner agencies and community organisations 

- engages partner agencies and community organisations in, and contributes to, wider 
developments 

• Extent to which partner agencies and community organisations report that service: 

- engages with, and supports, them in projects and developments 

- engages partner agencies and community organisations in, and contributes to, wider 
developments 

Key Features 

This indicator refers to the role of the service in promoting and encouraging effective 
partnership working with stakeholders, partner agencies and the local community. A central 
role of the service is to secure and sustain cohesive and meaningful partnerships. To encourage 
and support these processes the service will participate in bringing key partners together at the 
strategic decision-making level. To fulfil the requirements of Best Value and to conform to 
accepted good practice, there must be mechanisms in place to link leadership and 
management decisions to the needs of all stakeholders. This will require a range of approaches 
to consultation and communication that can be applied across the service's areas of activity. 
There should also be in place a public performance reporting (PPR) framework to 
communicate clearly with the full range of stakeholders and partner agencies and community 
organisations. 

The second theme deals with the views of partnership agencies and community organisations 
as reported in responses to questionnaires, surveys, focus groups and its unsolicited comments. 
These responses provide evidence of the extent of their satisfaction with the services and may 
cover aspects such as: 

• clarity of purpose and aims; 

• service level agreements, roles and remits; 

• access to services and facilities; 

• working across agencies and disciplines; 

• staff roles in partnerships; and 

• responsibilities to enquiries and complaints. 






QUALITY MANAGEMENT IN LOCAL AUTHORITY EDUCATIONAL PSYCHOLOGY SERVICES - SELF-EVALUATION FOR QUALITY IMPROVEMENT 






QUALITY MANAGEMENT IN LOCAL AUTHORITY EDUCATIONAL PSYCHOLOGY SERVICES - SELF-EVALUATION FOR QUALITY IMPROVEMENT 


How good is our management? 

(KEY AREA 8: PARTNERSHIPS AND RESOURCES) 

Ql 8.2 Financial management 

Themes: 

• Budget management and enterprise in securing funding 

• Range and implementation of financial procedures and controls 

• Processes for collecting, analysing and evaluating financial information 

• Providing Best Value 

Key Features 

This indicator relates to the capacity of the service to help to deliver planned national and local 
priorities. This will require the service to have developed a rigorous, thorough and imaginative 
approach to financial management. Such an approach will require high level managerial 
planning and control coupled with rigorous operational management, monitoring and review 
of a wide range of financial systems and procedures. The working relationship between the 
service and the authority's finance department will be critically important. The indicator also 
relates to the service's capacity to deliver Best Value through its approach to financial 
management. 
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How good is our management? 

(KEY AREA 8: PARTNERSHIPS AND RESOURCES) 

Ql 8.3 Resource management 

Themes: 

• Accommodation 

• Resources and equipment 

• Efficiency and effectiveness in use of resources 

• Health and safety 

Key Features 

The management of finances is closely linked to the wider management of resources. This 
indicator sets out to evaluate the effectiveness of the service's approach to the management of 
its resources in the widest sense. The service will need to demonstrate that it is fully aware of 
the range of resources at its disposal, from both internal and external sources, and that it has 
developed a planned, proactive approach to resource management. This provides a firm base 
from which to promote service development and continuous improvement. 
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How good is our management? 

(KEY AREA 8: PARTNERSHIPS AND RESOURCES) 

Ql 8.4 Information systems 

Themes: 

• Data collection, storage and retrieval 

• Linkages between, and sharing of, information 

• Processes for analysing, evaluating and using information 

Key Features 

A management information system (MIS) is a system used to enter, store, manipulate and 
retrieve information. A well-run, computerised management and administration system 
supports the key business processes of a modern service and provides the means of monitoring 
provision and improving effectiveness. An MIS should be seen in the context of helping a 
service to deliver its core functions and achieve its improvement objectives. Analyses of the 
data can give managers, staff and stakeholders a more informed view of current practice in the 
service. Because data sharing is a central aspect of the MIS, security is clearly important. 
Different levels of security are appropriate for different user groups. 
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How good is our leadership? 

(KEY AREA 9: LEADERSHIP) 

Ql 9.1 Vision, values and aims 

Themes: 

• Appropriateness and coherence with corporate and community vision, values and aims 

• Sharing and sustaining the vision 

• Promotion of positive attitudes to social and cultural diversity 

Key Features 

This indicator relates to the collegiate leadership of the service and to the way in which it 
exercises its functions through unity of purpose. This will be expressed through the service's 
effectiveness in establishing direction through its vision. To demonstrate good practice, the 
service will have to ensure that this vision actively influences practice at the point of delivery. 
This indicator focuses on the extent to which vision, values and aims guide planning for, and 
impact on, maintaining and improving the quality of services for children, young people, 
families and other stakeholders. 



A 
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How good is our leadership? 

(KEY AREA 9: LEADERSHIP) 

Ql 9.2 Leadership and direction 

Themes: 

• Strategic planning and communication 

• Strategic deployment of resources 

• Evaluation of risk 

Key Features 

This indicator is fundamentally about strategic planning for future sustainable development. It 
focuses on the mapping out of future developments which are challenging, realisable and 
sustainable. This indicator also relates to the success of senior managers in linking the 
authority's vision to strategic deployment of resources to deliver services, secure Best Value and 
manage sustainable development. Significant current and planned service activities will take 
place within a culture which supports and enables effective risk management. 



A 
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How good is our leadership? 

(KEY AREA 9: LEADERSHIP) 

Ql 9.3 Developing people and partnerships 

Themes: 

• Development of leadership capacity 

• Building and sustaining relationships 

• Teamwork and partnerships 

Key Features 

This indicator relates to the effectiveness of the service in building capacity for leadership at all 
levels and securing positive working relationships and successful outcomes with stakeholders 
and partner agencies. The indicator relates to the ethos and culture of the service and beyond, 
through its operations corporately within the authority and in joint working with its partner 
agencies. The effectiveness of the service management team, their deployment, responsibilities 
and co-working in relation to organisational requirements and key strengths are relevant. 
Delegation to and empowerment of staff and partners are important features, alongside the 
development and support of effective teamwork. 
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How good is our leadership? 

(KEY AREA 9: LEADERSHIP) 

Ql 9.4 Leadership of change and improvement 

Themes: 

• Support and challenge 

• Creativity, innovation and step change 

• Continuous improvement 

Key Features 

This indicator is concerned with the effectiveness of the leadership of the service to maintain 
high levels of quality, deliver continuous improvement, and work towards achieving excellence 
in the quality of applied psychological services for children, young people, families and other 
stakeholders. A critically important component of the leadership function is the need for 
service managers and stakeholders to challenge staff continuously to improve the quality of 
service, by setting demanding but realistic performance targets and by providing high-level 
support to assist them to achieve these. The indicator also relates to the ability and success of 
the service management team systematically to encourage and support innovative and 
effective practices which bring about positive step changes in learners' experiences. 
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Introduction 

The self-evaluation framework is about improving educational psychology services to deliver 
better outcomes for all service users. Schools have received a copy of How good is our school? 
The journey to Excellence, which outlines ten dimensions of excellence. These do not cover 
everything which goes on in a school but rather focus on those aspects which have a direct 
impact on learning and outcomes for all young people. The ten dimensions and their related 
features provide an indication of some of the things which make a school excellent. The ten 
dimensions and the related features are also relevant to educational psychology services and 
their journey to excellence. Educational psychology services can support services and schools 
on their journeys to excellence and also examine where the service is on its own journey to 
excellence. For example, schools may ask the educational psychology service for help in 
working on the partnership dimension. 


The ten dimensions of exceiience 



SUCCESSFUL 

LEARNERS 


CONFIDENT 

INDIVIDUALS 


RESPONSIBLE 
CITIZENS ■ 


EFFECTIVE 

CONTRIBUTORS 


FOCUSES ON OUTCOMES AND 
MAXIMISES SUCCESS FOR 
ALL LEARNERS 


WORKS IN 
PARTNERSHIPS WITH 
OTHER AGENCIES 
AND ITS COMMUNITY 


PROMOTES 
WELL-BEING AND 
RESPECT 


DEVELOPS A 
COMMON VISION 
AMONG CHILDREN 
AND YOUNG PEOPLE, 
PARENTS AND STAFF 


FOSTERS HIGH 
QUALITY LEADERSHIP 
AT ALL LEVELS 


VALUES AND 
EMPOWERS ITS 
STAFF AND YOUNG 
PEOPLE 


REFLECTS ON ITS 
OWN WORK 
AND THRIVES 
ON CHALLENGE 


DEVELOPS A 
CULTURE OF 
AMBITION AND 
ACHIEVEMENT 


WORKS TOGETHER 
WITH PARENTS TO 
IMPROVE LEARNING 
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Appendix II Grid for cross-referencing sources of evidence with quality indicators 
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GRID FOR CROSS REFERENCING 

SOURCES OF EVIDENCE WITH Qls 


SOURCE OF INFORMATION - DOCUMENTATION 


SERVICE DOCUMENTS 

Service development plan 

Statement of vision, values, aims 

Service handbook 

Standardised paperwork systems/proformas for use by educational psychologists 

Service policy and procedures 

Service guidelines on policies and procedures 

Service recruitment procedures, including paperwork 

Service induction procedures 

Written consultation framework for policies and procedures 

Portfolio of activities offered by the service 

Service Best Value report 

Research reports 

Service standards and quality report 

Collation of information from service feedback 

Summary of service achievements 

Summary of service evaluation on delivery of key services 

Summary of performance data with built-in evaluation 

Self-evaluation data/statistics 

Information for stakeholders, e.g. leaflets, website, publications 

Information leaflets available in Braille, other languages and on CD/DVD 

Records kept of work carried out during school visits 

Records within case files 

Formal reports on specific initiatives 

Collated feedback from training provided 

Letters of recognition of quality of service provided 

Records of complaints and outcomes 

Records kept on use of interpreter services 

Statements/certificates of competence in educational testing 
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Database of training materials 

Service publications 

Minutes of service business meetings 

Minutes of service development meetings 

Minutes of meetings for individual children/young people 

Records of issues regarding staff support 

Service log of delivery of training at local and national level 

Minutes from steering/planning groups 

Minutes of meetings with authority reps to plan service priorities 

Points for action from focus groups of service users 

Questionnaires for feedback on service delivery 

Structured interview schedule for feedback on service delivery 

Self-evaluation documentation 

Individual CPD logs kept by each educational psychologist 

Staff review and development summaries 

Evidence of equal opportunities monitored in recruitment and employment 

Published articles in press/journals/newsletters 

Practice level agreements with establishments 

Practice level agreements with other agencies 


AUTHORITY DOCUMENTS 

Authority development plan 

Authority evaluation reports 

Other authority documentation (name) 

Individual establishment development plans 

Paperwork outlining purposes of steering/planning groups 

Authority policy that is based on Council and BPS policy 

Integrated children's services plan 


OTHER DOCUMENTS 

National priorities for education 

HMIE reports 
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